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DETAILED ACTION 



Response to Amendment 

1 . The amendments, filed on 02/20/2009, have been entered and made of record. Claims 1-46 
are pending. 



Response to Arguments 
2. Applicant's arguments with respect to claims 1 -46 have been fully considered but are moot in 
view of the new ground(s) of rejection. 



Claim Rejections - 35 USC§102 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

4. Claims 1-2, 7-8, 10, 18-19, 22, 24-25, 30-31, 33, 42, and 45 are rejected under 35 
U.S.C. § 102(b) as being anticipated by Lelong, US 5,444,478. 

Regarding claim 1, Lelong discloses an apparatus (the image pickup system) for creating, 
managing and publishing interactive virtual tours (6:30-39), the apparatus comprising: 
a panorama data acquisition unit (three fixed cameras CI, C2, C3, Fig 2) 
implementing means of capturing panoramic data representing a plurality of 
environment maps and preparing said panoramic data for further processing 
(6:30-39); 
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a transform engine (an image reconstruction system 100, Fig 2) responsive to said 
panoramic data and implementing means of correcting distortions in said 
panoramic data and/or performing automatic, manual or interactive calibration of 
said panoramic data and/or transforming said panoramic data into a desired 
format or set of formats when necessary (10:28-48), wherein said distortion 
correction is accomplished through steps of: a) loading data including reference 
data that could be used to derive panoramic imaging system distortion profiles; b) 
specifying a linear or other predetermined distortion function or set of distortion 
functions and displaying representations of said distortion function or set of 
distortion functions; c) using distortion function or set of distortion functions to 
build a distortion profile for the panoramic imaging system that was used to 
acquire the reference data; d) performing transformation on panoramic data using 
distortion profile specified in c) and displaying results of said transformation; e) 
determining whether distortion is satisfactorily corrected; f) continuing with steps 
g) and h) if distortion is deemed not to have been satisfactorily corrected and 
continuing with step i) if distortion is deemed to have been satisfactorily 
corrected; g) automatically or interactively modifying the distortion function or 
set of distortion functions using feedback from displayed distortion function 
representation or transformation results obtained in step d) or by using data loaded 
in step a); h) repeating steps c) to g) until distortion is judged to have been 
satisfactorily corrected; i) storing distortion profile obtained in step c) for use in 
performing transformations on panoramic data acquired using the panoramic 
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imaging system for which data was loaded in step a) (3:19-42, please see section 
III, IV and V for image reconstruction processing steps in 9:26-12:12, and claims 

1-2); 

- a package generator (an address computer 200, Fig 3) adapted to generate virtual 
tour packages containing said panoramic data, commands and/or optional virtual 
tour data (12:13-14:12); 

- a viewing engine (an address computer 200, Fig 3) responsive to said panoramic 
data and/or virtual tour packages and implementing means for perspective 
correction, and user interaction with, said panoramic data and/or virtual tour data 
when necessary (12:13-14:12); 

a control engine (a control system 1, Fig 2) adapted to facilitate a higher level of 
interaction with said panoramic data and/or virtual tour data, wherein said control 
engine is connected operatively to and communicates bi-directionally with said 
viewing engine, renders representative information about all or parts of said 
virtual tour, permits a particular portion to be selected from said virtual tour and 
sends signals to said viewing engine that cause said viewing engine to permit the 
interactive navigation of said selected portion of said virtual tour, wherein said 
control engine also indicates or causes to be indicated what portion of said virtual 
tour is currently selected and what sub-part of said selected portion of said virtual 
tour is currently rendered, wherein said control engine is responsive to user input 
and/or commands from said viewing engine and is in turn capable of modifying 
said representative information about all or parts of said virtual tour in response to 
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said user input and/or said commands from said viewing engine and is further 
capable of communicating information indicative of such externally induced 
modifications to said user and/or said viewing engine (4:41-62, 10: 17-54, 1 1 :5- 
19); 

a display means (display means 10, Fig 2) for rendering output of said viewing 
engine, control engine, package generator, transform engine, and/or panoramic 
data acquisition unit (10:41-54). 
Regarding claim 2, Lelong teaches a panoramic data acquisition unit (three fixed cameras 

CI, C2, C3, Fig 2) is adapted to capture data representing panoramic images or video (6:30- 

39). 

Regarding claim 7, Lelong teaches a panoramic data acquisition unit (three fixed cameras 
CI, C2, C3, Fig 2) is adapted to capture each complete panoramic data block in a single 
image/video frame (6:30-39). 

Regarding claim 8, Lelong teaches a panoramic data acquisition unit (three fixed cameras 
CI, C2, C3, Fig 2) is adapted to capture each complete panoramic data block in a plurality of 
image/video frames that are combined to form complete panoramic data blocks (6:30-39). 

Regarding claim 10, Lelong teaches a panoramic data acquisition unit (three fixed 
cameras CI, C2, C3, Fig 2) further comprises of preparing and transferring all or portions of 
acquired panoramic data to transform engine, viewing engine, or display said control engine 
or said display; wherein said preparation and/or transfer involves compression and/or 
decompression of all or portions of acquired panoramic data; wherein said preparation and/or 
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transfer is in response to user commands or information inferred from the state of said 
viewing engine or said control engine (6:30-39). 

Regarding claim 18, Lelong teaches an image transformation engine (an image 
reconstruction system 100, Fig 2) being responsive to user input and implements for 
perspective correction of panoramic data (6:30-39). 

Regarding claim 19, Lelong discloses the viewing engine (an address computer 200, Fig 
3) further implements of navigating panoramic data (12:13-14:12). 

Regarding claim 22, Lelong teaches the representative information rendered by control 
engine (a control system 1 , Fig 2) about all or parts of virtual tour is rendered in 2D, 3D or 
higher dimensional space and/or time (10:28-44). 

Regarding claims 24-25, 30-31, 33, 42, and 45, these nine claims are method claims 
corresponding to apparatus claims 1-2, 7-8, 10, 19, and 22. Therefore, method claims 24-25, 
30-31, 33, 42, and 45 are analyzed and rejected as previously discussed with respect to 
apparatus claims 1-2, 7-8, 10, 19, and 22. 

Claim Rejections - 35 USC §103 

5. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

6. Claims 5 and 28 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Lelong, 
US 5,444,478, in view of Tanida, US 7,009,652. 

Regarding claims 5 and 28, note the discussion of Lelong above. Lelong does not 
mention comprises at least one grid of one or more photosensitive elements on a surface 
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with a spherical geometry. In same field of endeavor (video camera), Tanida teaches a well- 
known video camera which has a multiplicity of photosensitive pixel elements array on a 
spherical surface (2: 22-24). It would have been obvious to one of the ordinary skill in the 
art at the time the invention was made to have provided a multiplicity of photosensitive 
elements placed on spherical surface as taught by Tanida to the camera or acquisition unit of 
Lelong because the arrangement of photosensitive elements on spherical surface of Tanida 
would obtain a high resolution image (2:1 1-24, Tanida). 

7. Claim 6 and 29 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Lelong, US 
5,444,478, in view of Keast, US 5,721,585. 

Regarding claims 6 and 29, note the discussion of Lelong above. Lelong discloses the 
field of view of all of the cameras collectively captured is a complete 360-degree panoramic 
field of view. Lelong, however, does not specifically disclose a vertical field of view that is 
usually less than 180 degrees. Keast teaches an image system provides both a 180-degree 
vertical view and a 360-degree azimuthal view (5:16-28, and 6:54-67). It would have been 
obvious to one of ordinary skill in the art at the time invention was made to have added the 
180-degree vertical view as taught by Keast to the panoramic field of view of Lelong 
because both 180-degree vertical field of view and 360-degree panoramic field of view 
would provide a complete and uniform imaging of the sphere of view (5:16-29, Keast). 

8. Claims 9, 14-16, 20-21, 23, 32, 37-39, 43-44, and 46 are rejected under 35 U.S.C. § 103(a) 
as being unpatentable over Lelong, US 5,444,478, in view of Jackson et al, US 5,990,941 . 

Regarding claim 9, note the discussion of Lelong above. Lelong discloses an imaging 
system for the production of virtual tour data includes in either a still image or a video format 
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(2:9-1 1). Lelong, however, does not specifically disclose to resources on local and distributed 
networks. Jackson teaches the ability to delivery these image data, still or full motion video 
via broadcast, cable, digital network such as Internet, or on transportable media such as CD- 
ROM, videotape, or laserdisc from a remote site 12 to a local site 14. It would have been 
obvious to one of ordinary skill in the art at the time invention was made to have added 
ability to delivery these image data as taught by Jackson to the imaging system of view of 
Lelong because this feature provides the spherical image can be transmitted to multiple users 
at local locations (10:12-27, Jackson). 

Regarding claim 14, Jackson teaches a package generator implement specifying active 
regions on panoramic data. Jackson teaches a so-called "hot spots", as disclosed in column 4, 
lines 3-7, that are identified by a user as the user interacts with the stored data files 
representing the images. 

Regarding claim 15, Jackson teaches to specify the navigable paths for interactive display 
of any portion of a spherical image, as disclosed in column 4, lines 8-10. 

Regarding claim 16, Jackson teaches that the end user could utilize the navigable paths or 
walk-through sequences are navigated in automatic or guided mode (i.e. controls the 
direction of viewing by moving a computer mouse; see Figure 9 and also 8:16-31). 

Regarding claims 20 and 21, Jackson teaches a viewing engine implements means for a 
single user or a plurality of users to independently and simultaneously interact with and 
navigate panoramic data (8:16-31 and 10:12-27). 

Regarding claim 23, Jackson teaches the view engine and control engine comprising a 
unitary structure (microcomputer and control interface 15 as shown in Figure 1). 
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Claims 32, 37-39, and 43-44 are method claims. Thus, they are analyzed as previously 
discussed with respect to the apparatus claims 9, 14-16, and 20-21 above. 
9. Claims 11-13 and 34-36 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Lelong, US 5,444,478, in view of Xiong, US 5,960,108. 

Regarding claim 11, note the discussion of Lelong above. Lelong teaches the transform 
engine of claim 1, but doesn't teach conversion from one format. Xiong teaches the 
capability to convert the panoramic data from a first format to one or more formats and/or 
vice versa (i.e. conversion of 3D rays to 2D image, see Figure 2 and also 4:34-45). It would 
have been obvious to one of the ordinary skill in the art at the time the invention was made to 
have included the panoramic data conversion as taught by Xiong to the transform engine of 
Lelong because it allows an arbitrarily large field of view (4:54- 56, Xiong). 

Regarding claim 12, Xiong teaches that the panoramic data conversion is between 
different coordinates (an ordinary rectilinear lens is projection from a 3D ray to a 2D image 
position; 3:12-24, 4:34-67, and 6:1 to 7:26). 

Regarding claim 13, Xiong provides a plurality of images, each of the plurality of images 
including a plurality of parameters and optimizing the plurality of parameters for each of the 
plurality of images in accordance with a set of distortion functions that is based on a set of 
polynomials of suitable degree (i.e. r = c { 0 + c 2 02 + c 3 0 3 + , where the order of polynomial 
can be determined experimentally, see 4: 34-67). 

Regarding claims 34-36, these claims are method claims and analyzed as previously 
discussed with respect to the apparatus claims 11-13 above. 
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10. Claims 17 and 40 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Lelong, 
US 5,444,478, in view of Smith, US 6,226,658. 

Regarding claims 17 and 40, Smith teaches a layout code tuning in universally readable 
document files. More specifically, as disclosed in column 2, lines 28 - 46, Smith discloses a 
system and method in which page layout code in so-called "universal" document file formats 
is customized to make the file format more universally compatible across a spectrum of 
different manufacturers. It would have been obvious to one of the ordinary skill in the art at 
the time the invention was made to have included the universally readable document files as 
taught by Smith to the imaging system of Lelong because this feature makes it possible for 
arbitrary types and numbers of elements to be managed by universal file format. 

Allowable Subject Matter 

11. Claims 3-4, 26-27, 41 and 46 arc previously objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

- Ritchey (US 5,130,794) discloses A panoramic image based virtual reality display 
system includes a panoramic optical assembly, preferably of substantially spherical 
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coverage, feeding composite optical images to a light sensitive surface of a video 
camera for storage or further processing in image processing circuitry; 
- Dorbie (US 6,545,685) discloses A method for implementing edge blending between 
a first and second video frame to create a seamless multichannel display system; 
Henley (US 5,657,073) discloses a seamless multi-camera panoramic imaging with 
distortion correction and selectable field of view; and 

Zimmermann (US 5,185,667) discloses a device for omnidirectional image viewing 
providing pan-and-tilt orientation, rotation, and magnification within a hemispherical 
field-of-view which produces a circular image of an entire hemispherical field-of- 
view, which can be mathematically corrected using high speed electronic circuitry. 
13. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until 
after the end of the THREE-MONTH shortened statutory period, then the shortened statutory 
period will expire on the date the advisory action is mailed, and any extension fee pursuant to 
37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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14. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on 571-272-7564. The fax phone number for the 
organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from cither Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 

/Sinh N Tran/ 

Supervisory Patent Examiner, Art Unit 2622 
KW 

26 February 2009 



